1. Field of the Invention
The present invention relates to a sparse comparison neural network which compares a known input to an unknown input, and a method of comparing a known input to an unknown input.
2. Description of the Related Art
A neural network is an information processor which simulates biological neural functions in a computer. Neural networks may be used for visual pattern recognition. A type of neural network currently utilized for pattern recognition is a feedforward network which is trained using back-propagation techniques. Such a network is described in Parallel Distributed Processing, Volume I: Foundations, by D. E. Rumelhart et al. (MIT Press, 1987).
A conventional, three-layer, fully-connected, feedforward neural network trained using back-propagation techniques is shown generally at 10 in FIG. 1. The feedforward neural network as shown in FIG. 1 comprises an input layer 12, a hidden layer 14 and an output layer 16. Input layer 12 receives an input. Input layer 12 includes a plurality of artificial neurons, or nodes, 18. Hidden layer 14 receives inputs from input layer 12, and a weight is associated with each of the inputs to the hidden layer. Hidden layer 14 comprises a plurality of artificial neurons, or processing nodes, 20. A plurality of interconnections 22 distributes information from each node 18 in input layer 12 to each processing node 20 in hidden layer 14. Output layer 16 receives inputs from hidden layer 14, and a weight is associated with each of the inputs to the output layer. Output layer 16 also comprises a plurality of artificial neurons, or processing nodes, 24. Nodes 18 in input layer 12 are of a passive nature compared to processing nodes 20 and 24 in hidden layer 14 and in output layer 16, respectively. Processing nodes 20 and 24 perform a linear or non-linear transformation. A plurality of interconnections 26 distributes the information processed by each processing node 20 in hidden layer 14 to each processing node 24 in output layer 16. In each of the input and hidden layers, one of the nodes is fixed to a constant value and operates with its associated weight as an effective bias.
The weights associated with the inputs to the hidden layer and the output layer are derived during a training process. When the weights have been set to correct levels, a complex stimulus pattern at the input layer is sent to the hidden layer, resulting in an output pattern. The network is trained by feeding it a succession of input patterns and corresponding expected output patterns. The network "learns" by measuring the difference, at each output element, between the expected output pattern and the pattern that it just produced. Having done this, the weights are modified by a learning algorithm to provide an output pattern which more closely approximates the expected output pattern, while minimizing the error over the spectrum of input patterns.
While pattern recognition using a feedforward neural network trained using back-propagation techniques is known, successful applications in areas with large input arrays, e.g., in image processing and machine vision, are hindered by the fact that fully-connected, feedforward networks require large numbers of processing elements and weights in the hidden layer and the output layer for distributing information. Even though large back-propagation networks are feasible to build in principle, the major shortcoming of such networks is the training time required. The back-propagation training time for practically sized imaging feedforward networks can rapidly approach geological time periods on the largest supercomputers. Thus, the need exists for reducing the training time in neural networks having large input arrays.
Moreover, a back-propagation training algorithm for a known neural network can only be fed very rudimentary information for estimating feedback error. The teacher may give only a simple code, corresponding to a specific pattern that is being learned, at the network output. Therefore, the need also exists for developing a feedforward neural network trained by a back-propagation algorithm where considerably more information can be processed.
Accordingly, it is an object of the present invention to decrease the required network size, both in number of processing elements and in the number of connections between elements, compared to known feedforward neural networks of similar classification capacity.
In addition, it is an object of the present invention to provide a neural network for performing pattern recognition which significantly decreases the back-propagation training time compared to known networks of equivalent size which are applied to identical pattern recognition problems.
Additional objects and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objects and advantages of the invention may be realized and obtained by means of the instrumentalities and combinations particularly pointed out in the appended claims.